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EXPONENTIAL INEQUALITIES FOR SELF-NORMALIZED 
MARTINGALES WITH APPLICATIONS 

By Bernard Bercu and Abderrahmen Touati 

Universite Bordeaux 1 and Faculte des Sciences de Bizerte 

We propose several exponential inequalities for self-normalized 
martingales similar to those established by De la Pena. The keystone 
is the introduction of a new notion of random variable heavy on left or 
right. Applications associated with linear regressions, autoregressive 
and branching processes are also provided. 

1. Introduction. Let (M n ) be a locally square integrable real martingale 
adapted to a filtration F = (JP" n ) with Mq = 0. The predictable quadratic 
variation and the total quadratic variation of (M n ) are respectively given by 



(M) n = VjE[AM fc V fe _ 1 ] and [M] n = AM| 



k=l k=l 



where AM n = M n — M n _\. The celebrated Azuma-Hoeffding inequality 
[4, 16, 18] is as follows. 

Theorem 1.1 (Azuma-Hoeffding's inequality). Let (M n ) be a locally 
square integrable real martingale such that, for each 1 < k < n, < AM^ < 
bk a.s. for some constants at < bk- Then, for all x > 0, 

2x 2 



(1.1) P(|M n | >x) <2exp 



Yl=i{bk-a k y 



Another result which involves the predictable quadratic variation ((M) n ) 
is the so-called Freedman inequality [13]. 

Theorem 1.2 (Freedman's inequality). Let (M n ) be a locally square 
integrable real martingale such that, for each 1 < k < n, \AMk\ < c a.s. for 
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some constant c > 0. Then, for all x,y > 0, 
(1.2) F(M n > x, (M) n <y)< exp 

Over the last decade, extensive study has been made to establish expo- 
nential inequalities for (M n ) relaxing the boundedness assumption on its 
increments. On the one hand, under the standard Bernstein condition that 
for n > 1, p > 2 and for some constant c > 0, 

^E[|AM fe n^_ 1 ]<l-^(M) n , 
k=i z 

Pinelis [21] and De la Peha [8] recover (1.2). Van de Geer [12] also proves 

(1.2) replacing {M) n by a suitable increasing process. On the other hand, if 
(M n ) is conditionally symmetric which means that for n > 1, the conditional 
distribution of AM n given J- n -\ is symmetric, then De la Peha [8] establishes 
the nice following result. 

Theorem 1.3 (De la Peha's inequality). Let (M n ) be a locally square 
integrable and conditionally symmetric real martingale. Then, for all x,y > 
0, 

(1.3) P(M n > x, [M] n <y)< exp 

Some extensions of the above inequalities in a more general framework 
including discrete-time martingales can also be found in [9, 11] where the 
conditionally symmetric assumption is still required for (1.3). We also refer 
the reader to the recent survey of De la Peha, Klass and Lai [10]. 

By a careful reading of [8], one can see that (1.3) is a two-sided exponential 
inequality. More precisely, if (M n ) is conditionally symmetric, then, for all 
x,y> 0, 

(1.4) P(|M n |>x,[M]„<y)<2exp 

By comparing (1.4) and (1.1), we are only halfway to Azuma-Hoeffding's 
inequality which holds without the total quadratic variation [M] n . 

The purpose of this paper is to establish several exponential inequalities 
in the spirit of the original work of De la Peha [8]. In Section 2, we shall 
propose two-sided exponential inequalities involving (M) n as well as [M] n 
without any assumption on the martingale (M n ). Section 3 is devoted to the 
introduction of a new concept of random variables heavy on left or right. 
This notion is really useful if one is only interested in obtaining a one-sided 



2(y + cx)J' 
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exponential inequality for (M n ). It also provides a clearer understanding of 
De la Pena's conditional symmetric assumption. We shall show in Section 
4 that this new concept allows us to prove (1.3). As in [8], we shall also 
propose exponential inequalities for (M n ) self-normalized by [M) n or {M) n . 
Section 5 is devoted to applications on linear regressions, autoregressive and 
branching processes. All technical proofs are postponed to the Appendix. 



2. Two-sided exponential inequalities. This section is devoted to two- 
sided exponential inequalities involving {M) n and [M] n . We start with the 
following basic lemma. 



Lemma 2.1. Let X be a square integrable random variable with mean 
zero and variance a 2 > 0. For all t & R, denote 



(2.1) L{t)=E 
Then, we have for all t £ K, 



(2.2) 



tr 

expl tX X' 



L(i)<l + |a 2 . 



Proof. The proof is given in Appendix A. □ 



Our first result, without any assumption on (M n ), is as follows. 



Theorem 2.1. Let (M n ) be a locally square integrable martingale. Then, 
for all x, y > 0, 

(2.3) P(|M n | > x, [M] n + (M) n <y)< 2exp 

Remark 2.1. A similar result for continuous-time locally square inte- 
grable martingale may be found in the first part of Proposition 4.2.3 of 
Barlow, Jacka and Yor [5]. 




For self-normalized martingales, we obtain the following result. 



Theorem 2.2. Let (M n ) be a locally square integrable martingale. Then, 
for all x, y > 0, a > and b > 0, 

i2A) p ( Jwu £ X ' {M) " £ m " +y ) 5 *->{-*{*>+%))■ 
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Moreover, we also have 



(2.5) 



\M„ 



a + b(M) r 



> x, [M] n < y{M) n 



< 2 inf f E 

p>i\ 



exp 



(-(^- i )(i^)(« 6 4< M >- 



i/p 



Proof. The proof is given in Appendix B. □ 

Remark 2.2. It is not hard to see that (2.4) and (2.5) also hold ex- 
changing the roles of (M) n and [M] n . 

3. Random variables heavy on left or right. This section deals with our 
new notion of random variables heavy on left or right. It allows us to improve 
Lemma 2.1. 

Definition 3.1. We shall say that an integrable random variable X is 
heavy on left if E[X] = and, for all a > 0, E[T a (X)] < where 

r a (X)=min(|X|,a)sign(X) 

is the truncated version of X . Moreover, X is heavy on right if —X is heavy 
on left. 

Remark 3.1. Let F be the cumulative distribution function associated 
with X. Standard calculation leads to E[T a (X)] = —H(a) where H is the 
function defined, for all a > 0, by 



H(a)= I" F(-x)-{l-F{x-))dx 
Jo 



where F(x—) stands for the left limit of F at point x. Consequently, X is 
heavy on left if E[X] = and, for all a > 0, H(a) > 0. Moreover, H is equal 
to zero at infinity as 

lim H(a) = -E\X) = 0. 

a— >oo 

Furthermore, one can observe that a random variable X is symmetric if and 
only if X is heavy on left and on right. 

The following lemma is the keystone of our one-sided exponential inequal- 
ities. 

Lemma 3.1. For a random variable X and for all t € M, let 



L(t)=E 



exp( tX- — X 2 
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(1) If X is heavy on left, then for all t>0, L(t) < 1. 

(2) If X is heavy on right, then for all t < 0, L(t) < 1. 

(3) If X is symmetric, then for all t L{t) < 1. 

Proof. The proof is given in Appendix A. □ 

We shall now provide several examples of random variables heavy on 
left. More details concerning these examples may be found in Appendix E. 
We wish to point out that most of all positive random variables centered 
around their mean are heavy on left. As a matter of fact, let Y be a positive 
integrable random variable with mean m and denote 

X = Y-m. 

Discrete random variables. 

(1) If Y has a Bernoulli distribution B(p) with parameter <p < 1, then 
X is heavy on left, heavy on right, or symmetric if p < 1/2, p> 1/2, or 
p = 1/2, respectively. 

(2) If Y has a Geometric distribution Q(p) with parameter < p < 1, then 
X is always heavy on left. 

(3) If Y has a Poisson distribution V(X) with parameter A > 0, then X is 
heavy on left as soon as 

W X k 
2exp(-A)£->l. 
fc=o K - 

One can observe that this condition is always fulfilled if A is a positive 
integer; see Lemma 1 of [1]. 

Continuous random variables. 

(1) If Y has an exponential distribution £ (A) with parameter A > 0, then 
X is always heavy on left. 

(2) If Y has a Gamma distribution Q(a, A) with parameters a, A > 0, then 
X is always heavy on left. 

(3) If Y has a Pareto distribution with parameters a, A > 0, that is, Y = 
aexp(Z) where Z has an exponential distribution £ (A), then X is always 
heavy on left. 

(4) If Y has a log-normal distribution with parameters met and cr 2 > 0, 
that is, Y = exp(Z) where Z has a Normal distribution J\f(m,a 2 ), then 
X is always heavy on left. 



G 
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4. One-sided exponential inequalities. Our next results are related to 
martingales heavy on left in the sense of the following definition. 

Definition 4.1. Let (M n ) be a locally square integrable martingale 
adapted to a filtration F = (J- n ). We shall say that (M n ) is heavy on left 
if all its increments are conditionally heavy on left. In other words, for all 
n > 1 and for any a > 0, E[T a (AM n )| J^ n _i] < 0. Moreover, (M n ) is heavy on 
right if (— M n ) is heavy on left. 

We shall recover Theorem 1.3 under the assumption that (M n ) is heavy 
on left. 



Theorem 4.1. Let (M n ) be a locally square integrable martingale heavy 
on left. Then, for all x,y > 0, 



(4.1) 



For self-normalized martingales, our results are as follows. 



Theorem 4.2. Let (M n ) be a locally square integrable martingale heavy 
on left. Then, for all x > 0, a > and b > 0, 



M n 



a + b[M] r 



> x ) < inf [ E 

p>i V 



(4.2) 

and, for all y > 0, 



(43) ~\a + b[M] r 
Moreover, we also have 



exp[-(p-l)x 2 [ab + —[M} n 



b 2 y 



> x, [M] n >y) < exp( -x 2 { ab + 



(4.4) 



M„ 



a + b(M) n 
< inf [ E 

p>i V 



> x, [M) n < y{M) r 



exp[-(p-l) — [ab+—(M) n 



i/p 



Proof. The proof is given in Appendix C. □ 



Remark 4.1. In the particular case p = 2, Theorem 4.2 is due to De 
la Peha [8] under the conditional symmetric assumption on (M n ). The only 
difference between (2.5) and (4.4) is that (1 + y) is replaced by y in the 
upper-bound of (4.4). 
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Remark 4.2. A locally square integrable martingale (M n ) is Gaus- 
sian if, for all n > 1, the distribution of its increments AM n given J- n -\ 
is A/"(0, A(M) n ). Moreover, (M n ) is called sub-Gaussian if there exists some 
constant a > such that, for all n > 1 and iGl, 



(4.5) 



E[exp(tAM n )|.F n _i] < exp 



a 2 t 2 



A{M) n 



It is well known that if the increments of (M n ) are bounded or if (M n ) is 
Gaussian, then (M n ) is sub-Gaussian. In addition, if (M n ) satisfies (4.5), 
then inequalities (4.1), (4.2) and (4.3) hold with appropriate upper-bounds, 
replacing [M] n by {M) n everywhere. For example, (4.2) can be rewritten as 



(4.6) 



a + b(M) n 

< inf ( E 

-p>i\ 



> x 



exp -(p- 1)- 



a- 



b 2 

ab + —(M), 



i/p 



5. Applications. 



5.1. Linear regressions. Consider the stochastic linear regression given, 
for all n > 0, by 



(5.1) 



n+l 



where X n , c/> n and the observation, the regression variable and the 

driven noise, respectively. We assume that (4> n ) is a sequence of indepen- 
dent and identically distributed random variables. We also assume that (e n ) 
is a sequence of identically distributed random variables, with mean zero 
and variance a 2 > 0. Moreover, we suppose that, for all n > 0, the random 
variable e n +i is independent of T n where T n = <r(0o, £i, • ■ • , 4>n-i,£n)- In or- 
der to estimate the unknown parameter 9, we make use of the least-squares 
estimator 9 n given, for all n > 1, by 



(5.2) 



2^k=i 9k-i 

It immediately follows from (5.1) and (5.2) that 
(5.3) 9 n - " 



a 



(M) n 



where 



n 

E 

fe=i 



4>k~ie k and (M) n = a 2 ^ 



fe=i 
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Let H and L be the cumulant generating functions of the sequences (0 2 ) 
and (e 2 ), respectively given, for all t € R, by 

H(t) = logE[exp(t<^)] and L(t) = logE[exp(te 2 )]• 

Corollary 5.1. Assume that L is finite on some interval [0, c] with 
c > and denote by I its Fenchel-Legendre transform on [0, c] , 

I(x) = sup {xt — L(t)}. 

0<t<c 

Then, for all n > 1, x > and y > 0, we have 

p(\e n -e\>x) 

Remark 5.1. Corollary 5.1 is also true if ((j) n ,£ n ) is a sequence of in- 
dependent and identically distributed random vectors of M 2 such that the 
marginal distribution of e n is symmetric. By use of (4.4), inequality (5.4) 
holds replacing (1 + y) by y in the argument of H. 

Remark 5.2. As soon as the sequence (£«) is bounded, the right-hand 
side of (5.4) vanishes since we may directly compare [M n ] with {M) n . For 
example, assume that (e n ) is distributed as a centered Bernoulli B(p) dis- 
tribution with parameter < p < 1. If r = max(p, q), we clearly have for all 
ra>0, 

r 2 

[M] n <—(M) n . 

pq 

Consequently, we immediately infer from (2.5) that for all n > 1 and x > 0, 
F(\6 n -6\>x)<2ew(jH(~y). 

Furthermore, assume that (<fi n ) is distributed as a normal M(0,t 2 ) distribu- 
tion with variance r 2 > 0. Then, we deduce that for all n > 1 and x > 0, 

/ 2 2 

n , i t x 



> x) < 2exp --log 1 + 



2r 2 



Proof of Corollary 5.1. It follows from (2.5) that, for all n > 1, 
x > and y > 0, 



>x)=p(\M n \ > ^(M) n j<P n (x,y) + Q n (y) 
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where Q n {y) =P([M] n > y(M) n ) and 



P n (x,y) = 2miU expf-(p-l) 



2 inf exp I — H 



p>i 



2a\l + y) 
(p-l)x 2 



(M) n 



2a 2 (l + y) 



In addition, for all y > and < t < c, 

Q n {y) <AjZ £ l> a2 y] < ex P (-a 2 iy)E 



\k=l 



< exp(—a 2 ty + nL{t)) < exp ^— nl 
which achieves the proof of Corollary 5.1. □ 



exp 



<J 2 y 
n 



el 



k=l 



5.2. Autoregressive processes. Consider the autoregressive process given, 
for all n > 0, by 

(5.5) X n+ i = 9X n + e n+ i 

where X n and e n are the observation and the driven noise, respectively. We 
assume that (e n ) is a sequence of independent and identically distributed 
random variables with standard A^(0, a 2 ) distribution where a 2 > 0. The 
process is said to be stable if \8\ < 1, unstable if \0\ = 1 and explosive if 
|#| > 1. We can estimate the unknown parameter 9 by the least-squares or 
the Yule- Walker estimators given, for all n > 1, by 



(5.6) 



YJk=i x 2 



and B r , 



k-l 



J2k=i Xk~\X k 

2^fc=0 A fc 



It is well known that 9 n and 9 n both converge almost surely to 9 and their 
fluctuations can be found in [22]. In the stable case \9\ < 1, the large devia- 
tion principles were established in [6]. More precisely, set 



9-VWT, 



and b ■■ 



9 + V& 2 T% 



4 4 
Assume that Xq is independent of (e n ) with AA(0,cr 2 /(l — 9 2 )) distribution. 
Then, (9 n ) and (9 n ) satisfy large deviation principles with good rate func- 
tions respectively given by 



1 



I{x) 
J(x) 



log 



1 + 9 2 - 29x 
T^x 2 



log|6»-2x|, 

l + 9 2 - 29x\ 



+oo, 



1 



if x £ [a, b], 
otherwise, 

if xe]-l,i[, 

otherwise. 
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It is only recently that sharpjarge deviation principles were established for 
the Yule-Walker estimator 9 n in the stable, unstable and explosive cases 
[7]. Much work remains to be done for the least-squares estimator 9 n . Our 
goal is to propose, whatever the value of 9 is, a very simple exponential 
inequality for both 9 n and 9 n . For the sake of simplicity, we assume that Xq 
is independent of (e n ) with A^(0,r 2 ) distribution where r 2 > a 2 . 



Corollary 5.2. For all n > 1 and x > 0, we have 
(5.7) ¥(\9 n -9\ >x) <2exp 



2 

nx 
2(1 +y x 

where y x is the unique positive solution of the equation h(y x ) = x 2 and h is 
the function h(x) = (1 + x) log(l + x) — x. Moreover, for all n>l and x > 0, 
we also have 

(5.8) ¥(\9 n -9\>x+\9\)< 2exp(- ^ ) . 

V 2(l + y x )J 

Proof. The proof is given in Appendix D. □ 

Remark 5.3. Inequality (5.7) can be very simple if x is small enough. 
As a matter of fact, one can easily see that for all < x < 1, h{x) < x 2 /4. 
Consequently, it immediately follows from (5.7) that, for all < x < 1/2, 

^ / nx 2 

P(W .-,I> I )< !1 ^- 5JI _ 5) _ 

Moreover, if 9 > 0, we can deduce from (5.6) that, for all x > 0, 



> x) < exp 



9 

nx 



2(1+2/*) 



5.3. Branching processes. Consider the Galton-Watson process starting 
from Xq = 1 and given, for all n > 1, by 

X n -1 

(5.9) X n = ]T Y nik 

k=l 

where (Y n ^) is a sequence of independent and identically distributed, non- 
negative integer- valued random variables. The distribution of (Y n ^), with 
finite mean m and variance <r 2 , is commonly called the offspring or repro- 
duction distribution. Hereafter, we shall assume that m > 1. In order to 
estimate the offspring mean m, we can make use of the Lotka-Nagaev or 
the Harris estimators given, for all n > 1, by 

(5.10) m n = ^7 and m n = = — . 

Atj-1 2^fc=l A fc-l 
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Without loss of generality, we can suppose that the set of extinction of the 
process (X n ) is negligible. Consequently, the Lotka-Nagaev estimator fh n 
is always well defined. It is well known that rh n and fh n both converge 
almost surely to m and their fluctuations are given in [3, 14, 15]. Moreover, 
the large deviation properties associated with (fh n ) may be found in [2, 19, 
20]. Our goal is now to establish, as in the previous sections, exponential 
inequalities for both fh n and fh n . Denote by L the cumulant generating 
function associated with the centered offspring distribution given, for all 
t € R, by L(t) = logE[exp(t(y nifc - m))]. 

Corollary 5.3. Assume that L is finite on some interval [— c, c] with 
c > and let I be its Fenchel-Legendre transform, 

I(x)= sup {xt — L(t)}. 

-c<t<c 

Then, for all n > 1 and x > 0, 

(5.11) P(|m n - m\ > x) < 2E[exp(- J(x)X n _i)] 

where J(x) = mm(I(x), I(—x)) . Moreover, we also have 



In addition, if S n = X)fc=o ^-k> we have for all n>l and x > 0, 



(5.12) P(|m n - ml > x) < 2 inf (E[exp(-(p - 1) J(x)X n _ 1 )}) 

p>i 



(5.13) P(\fhn -m\>x)<2 inf (E[exp(-(p - 1) J(x)S' n _i)]) 1/p . 

p>i 

Proof. The proof is given in Appendix E. □ 

Remark 5.4. On the one hand, inequality (5.12) obviously holds for the 
Harris estimator m n since we always have S n > X n . On the other hand, in 
order to specify the right-hand side of (5.11), (5.12) or (5.13), it is necessary 
to find an upper-bound or to provide an explicit expression of the moment 
generating function of X n . One can easily carry out this calculation when 
the offspring distribution is the geometric G(p) distribution with parameter 
< p < 1. As a matter of fact, in that particular case, the offspring mean 
m = l/p and it follows from formula (7.3) of [15] that for all < s < 1, 

E[s Xn ] < _P_f_. 



Consequently, for all n > 1 and x > 0, we obtain the simple inequality 

2p n exp(— J{x)) 



\m„ — m > x) < 



p(l - exp(-J(x))) 
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If the offspring distribution is not geometric, one can precisely estimate the 
moment generating function of X n using Theorem 1, page 80 of [3] which 
gives a good approximation of the distribution of X n based on the limiting 
distribution 

W = lim — — a.s. 



APPENDIX A 

This appendix is devoted to the proofs of Lemma 2.1 and Lemma 3.1. 
Lemma 2.1 immediately follows from Jensen's inequality. As a matter of 
fact, (2.1) implies that for all t 6 R, 



L(t) > exp ( E 



tX X' 



Consequently, we obtain that for all t € R, 



> exp( -—a' 



(A.l) 

Furthermore, for all t £ R, 
(A.2) L(t) + L{-t) = 2E. 



L(t)> 1 -i-a 2 



exp( --X 2 j cosh(tX) 



< 2 



by the well-known inequality cosh(x) < exp (x 2 /2). Hence, we obtain from 
(A.l) together with (A.2) that for all t £ R, 

t 2 

L{t) <2 - L(-t) <l + — a 2 . 

Lemma 3.1 is much more difficult to prove. Let / be the function defined, 
for all x £ R, by 



f(x) = exp I x 



We clearly have f'(x) = (1 — x)f(x) and f'(—x) = (1 + x)f(—x). We shall 
also make use of the functions a and b defined, for all x G R, by a(x) = /'(— x) 
and b(x) = f'(—x) — f'(x). One can realize that, for all x > 0, < a(x) < 1, 
< b(x) < 2 and a'(x) < as a'(x) = —(2x + x 2 )f(—x). After those simple 
preliminaries, we are in position to prove Lemma 3.1. For all t G R, 



L(t)=E 



exp tX 



t 2 X 2 



f(tx)dF(x) 
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where F is the distribution function associated with X. Integrating by parts, 
we have for all t G R, 



L(t) = -t / f'(tx)F(x)dx 



(A.3) 



-oo 
+00 



+00 



f'(tx)F(x)dx-t f'{tx)F{x)dx 



f'{-tx)F(-x)dx-t f'{tx)F{x)dx. 



Consequently, as 



t / f'(tx)dx 



exp tx 



t 2 x 2 



we obtain from (A.3) that, for all t € R, L(t) = 1 — tl(t) where 



lit) 



(A.4) 
with 



+00 



f'{-tx)F(-x)dx 
A(t) + B(t) 



+00 



f'(tx)(l- F(x))dx 



A(t) 



Bit) 



a(tx)(F(-x) - (1 - F(x))) dx, 



b(tx)(l- F(x))dx. 



First of all, assume that X is heavy on left. Our goal is to show that, for 
all t > 0, the integral I(t) is nonnegative. We obviously have, for all t > 0, 
B(t) > as b{tx) > 0. In addition, for any a > 0, let 



H(a)= r F{-x) - (l-F(x-))dx. 
Jo 



Since H'(a) = F(—a) — (1 — F(a)) almost everywhere, integrating once again 
by parts, we find that 



A(t) 



(A.5) 



a{tx)H'(x) dx = [a(tx)H(x) 



a'(tx)H(x) dx 



ta'(tx)H(x) dx 



as H (0) = and H vanishes at infinity. Hereafter, as X is heavy on left, 
H(a) > for all a > 0. Moreover, we recall that, for all x > 0, a'(x) < 0. 
Hence, we immediately deduce from (A.5) that, for all t > 0, A(t) > 0. Con- 
sequently, relation (A.4) leads to I(t) > and L{t) < 1 for all t > 0, which 
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completes the proof of part (1) of Lemma 3.1. Next, if X is heavy on right, 
— X is heavy on left. Hence, we immediately infer from (2.1) and part (1) of 
Lemma 3.1 that L(t) < 1 for all t < 0. Finally, part (3) of Lemma 3.1 follows 
from the conjunction of parts (1) and (2). Another straightforward way to 
prove part (3) is as follows. If X is symmetric, we have for all t G R, 

r r+oo 

L(t) = / f(tx) dF(x) = / (f(tx) + f(-tx)) dF(x) 
Jr Jo 

r+oo 

= 2 exp(-£ V/2) cosh(tx) dF(x) < 1 
Jo 

by the well-known inequality cosh(x) < exp(x 2 /2). 

APPENDIX B 

In order to prove Theorems 2.1 and 2.2, we shall often make use of the 
following lemma. 

Lemma B.l. Let (M n ) be a locally square integrable martingale. For all 
t G R and n > 0, denote 

V n (t) = exp(tM n - |-([M]„ + {M)rSj . 

Then, for all t G R, (V n (t)) is a positive supermartingale with K[V n (t)] < 1. 

PROOF. For all t G R and n > 1, we have 

/ t 2 

V n (t) = K-i(i) expUAM n - — (A[M]„ + A(M) n ) 

where AM n = M n - M„_i, A[M] n = AM 2 and A(M) n = E[AM^ n _!]. 
Hence, we deduce from Lemma 2.1 that for all tgl, 

t 2 \ / t 2 



Ef%(t)|^ n _i] < K-i(t)exp^--A(M)„J ^1 + -A(M), 

<V n -!(t). 

Consequently, for all t G R, (V n (t)) is a positive supermartingale such that, 
for all n > 1, E[V n (t)] < E[V n -i(t)) which implies that E[V n (t)] < E[Vb(t)] = 
1. □ 

We are now in position to prove Theorems 2.1 and 2.2 inspired by the 
original work of De la Peha [8]. First of all, denote 



Z n = [M] n + (M) 
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For all x, y > 0, let 

A n = {\M n \>x,Z n <y}. 

We have the decomposition A n = A+ U A~ where A+ = {M n > x,Z n < y} 
and A~ = {M n < —x, Z n <y}. By Markov's inequality, we have for all t > 0, 

P(A+)<E 



( ^M n -% )t A + 



< E 



exp 



ft r 

{- 2 Mn--Z n 



<exp(^-|yE[F n (i)]P(A+). 



Hence, we deduce from Lemma B.l that for all t > 0, 
(B.l) 



^+)<exp(^-|)v / F^4+). 



Dividing both sides of (B.l) by yF(At) and choosing the value t = x/y, we 
find that 



F(A+) < exp 



x 2 \ 

2y)' 



We also find the same upper-bound for F(A n ) which immediately leads to 
(2.3). 

We next proceed to the proof of Theorem 2.2 in the special case a = and 
6 = 1 inasmuch as the proof for the general case follows exactly the same 
lines. For all x,y > 0, let 

B n = {\M n \ > x(M) n , {M) n - [M) n >y} = B+ U B~ 

where 

B+ = {M n > x(M) n , (M) n - [M] n > y}, 
B~ = {M n < -x(M) n , (M) n - [M] n > y}. 
By Cauchy-Schwarz's inequality, we have for all t > 0, 

P(H+)<E 

1-2 



(B.2) 



ft tx 
exp{-M n --(M) n )l B + 



< E 



expQM n - l -Z^j expQ(t - 2x){M) n + j[M] r )jt B , 



Consequently, we obtain from (B.2) with the particular choice t = x that 
(B.3) F(B+)<exp(-^)J¥(J3+). 
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Therefore, if we divide both sides of (B.3) by J~P(Bn), we find that 



\B+) < exp 



2 

x y 



The same upper-bound holds for ¥(B n ) which clearly implies (2.4). Fur- 
thermore, for all x,y > 0, let 

C n = {\M n \ > x(M) n , [M] n < y(M) n } = C+ U C~ 

where 

C+ = {M n > x(M) n , [M] n < y(M) n }, 
C~ = {M n < -x(M) n , [M] n < y(M) n }. 
By Holder's inequality, we have for all t > and q > 1, 

P(C+)<E 



ft tx 
exp( -M n (M) n )l c , 



(B.4) 



< E 



t 



t 



exp( -M n - —Z n j exp(^— (i - 2x + ty)(M)„ ) 1^ 



< E 



exp(^(t-2x + ty)(M) 1 



i/p 



Consequently, as p/q = p — 1, we can deduce from (B.4) and the particular 
choice t = x/( 1 + y) that 



»(C+)<inf E 
p>i \ 



exp -{p- 1) 



2(1 + y) 



(M) r 



i/p 



We also find the same upper-bound for P(C n ) which completes the proof of 
Theorem 2.2. 



APPENDIX C 

The proofs of Theorems 4.1 and 4.2 are based on the following lemma. 

Lemma C.l. Let (M n ) be a locally square integrable martingale. For all 
t G R and n > 0, denote 

( t 2 
W n (t) = expfiM n - -[M], 

(1) If (M n ) is heavy on left, then for all t>0, (W n (t)) is a supermartingale 
with E[W„(t)] < 1. 

(2) If(M n ) is heavy on right, then for all t < 0, (W n (t)) is a supermartingale 
with E[W n (t)) < 1. 
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(3) If (M n ) is conditionally symmetric, then for all t E R, (W n (t)) is a 
supermartingale with M[W n (t)] < 1. 

Proof. Lemma C.l part (3) is due to De la Peria [8], Lemma 6.1. Our 
approach is totally different as it mainly relies on Lemma 3.1. Assume that 
(M n ) is heavy on left. For all t £ R and n > 1, we have 

/ t 2 
W n {t) = Wn-i(t) exp ( tAM n - —A[M] n 

where A[M] n = AM^. We infer from Lemma 3.1 part (1) that for all n > 1 
and for all t > 0, 



E 



exp[tAM n --AMZ 



< 1. 



Consequently, for all t > 0, (W n (t)) is a positive supermartingale such that, 
for all n > 1, E[W n (t)] < E[W n _i(t)] which leads to E[W n (t)] < E[W (t)] = 1. 
The rest of the proof is also a straightforward application of Lemma 3.1. □ 

By use of Lemma C.l, the proof of Theorem 4.1 is quite analogous to that 
of Theorem 2.1 and therefore is left to the reader. We shall proceed to the 
proof of Theorem 4.2 in the special case a = and 6=1. For all x > 0, let 
A n = {M n > x[M] n }. By Holder's inequality, we have for allt > and q > 1, 



P(A„) < E 
(C.l) <E 



ft tx \ 
exp -M n [M} n )l An 

ex P QM n - ^-[M] n J ex p(^(* " ^)[M]^± An 



< E 



exp 



g(t-2x)[M] n 



i/p 



(E[W n (t)])^. 



Since (M n ) is heavy on left, it follows from Lemma C.l that for all t > 0, 
E[W n (t)] < 1. Consequently, as p/q = p— 1, we can deduce from (C.l) and 
the particular choice t = x that 



¥(A n ) < inf ( E 

p>i v 



eX p(-(p-l) — [M} n 



i/ P 



Furthermore, for all x,y > 0, let B n = {M n > x[M] n , [M] n > y}. As before, 
we find that for all < t < 2x, 



< E 



. t 

exp -M n 



~T \M]n \ exp ( - A (t - 2x) [M] n ) t B . 



<exp(^j(t-2x^jE 
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< 



( x 2 y 

choosing the value t = x. Finally, the last inequality of Theorem 4.2 is left 
to the reader as its proof follows exactly the same arguments as (4.3). 

APPENDIX D 

We shall now focus our attention on the proof of Corollary 5.2. It imme- 
diately follows from (5.5) together with (5.6) that for all n > 1, 

where 

n n 

M n = Y J X k ^ 1 e k and (M) n = a 2 ]T X 2 k _ x . 

k=l k=l 

The driven noise (e n ) is a sequence of independent and identically distributed 
random variables with M(0, a 2 ) distribution. Consequently, for all n > 1, the 
distribution of the increments AM n = X n _\E n given T n -\ is M(0,cr 2 X 2 _ 1 ) 
which implies that (M n ) is a Gaussian martingale. Therefore, we infer from 
inequality (4.6) that for all n > 1 and x > 0, 



F(\9 n -9\>x)=P[ \M, t 

(D.2) 

< 2 inf ( IE 
p>iV 



x 2 \l\ Up 



exp (-(p-i)_(M) n 



Similar result may be found in [17, 23, 24]. We are now halfway to our goal 
and it remains to find a suitable upper-bound for the right-hand side of 
(D.2). For all t G R such that 1 - 2a 2 t > 0, if a = l/Vl -2a 2 t, we deduce 
from (5.5) that, for all n > 1, 

E[exp(tX2)|^ n _ 1 ] = exp(t9 2 X 2 _ 1 )E[exp(26tX n _ 1 e n + t£* n )\T n -i] 

exp^A^) f ( x 2 \ 
= 7= / exp - exp(2^A n _ix) dx. 

Hence, if /? = 2tacr0X n _i, we find via the change of variables y = x/aa that 

nrr f+Y*\\v i aex P( tg2x n-i) f ( y\ R \j 
E[exp(tJST n )|^ n _i] = y^exp^-y + /3y J dy 

= aexp(t9 2 X 2 _ 1 + = aexp(ta 2 6 2 Xl_ 1 ), 
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which implies that, for all t < and n > 1, 
(D.3) E[ex P (i^)|J^_i]<a. 

Furthermore, as X is N(0,t 2 ) distributed with r 2 > cr 2 , Efexp^X, 2 )] < a. 
It immediately follows from (D.3) together with the tower property of the 
conditional expectation that for all t < and n > 0, 



(D.4) E[exp(t(M) n )] < (1 - 2<r 4 i) 



-n/2 



Consequently, we deduce from the conjunction of (D.2) and (D.4) with the 
value t = — (p — l)x 2 /2cr 4 and the change of variables y = (p— l)x 2 that for 
all x > and n > 1, 

P(|0n " «| > x) < 2 inf exp (-—£{y) 
y>0 \ 2 

where the function £ is given by 

log(l + y) 



We clearly have 



t(y)- 

x z + y 



[l + y)(x 2 + y) 2 



where h(y) = (1 + y) log(l + y) — y. One can observe that the function h is 
the Cramer transform of the centered Poisson distribution with parameter 
1. Let y x be the unique positive solution of the equation h(y x ) = x 2 . The 
value y x maximizes the function I and this natural choice clearly leads to 
(5.7). Finally, it follows from (5.6) and (5.7) that for all x > and n > 1, 



(D.5) P(|# n -# + #/ n |>x)<2exp 



2 

nx 



2(1 + 2/*) 



where the random variable < f n < 1. Hence, (D.5) implies (5.8) which 
completes the proof of Corollary 5.2. 



APPENDIX E 

We shall now proceed to the proof of Corollary 5.3. We only focus our 
attention on the Harris estimator inasmuch as the proof for the Lotka- 
Nagaev estimator follows essentially the same lines. First of all, relation 
(5.9) can be rewritten as 



(E.l) 



X n = mX n -i + i n 
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where £ n = X n — E[X n |.F n _i]. Consequently, we obtain from (5.10) together 
with (E.l) that for all n > 1, 



m r , 



m ■ 



Mn 
Sn— 



where M n = 



k=l 



Moreover, for all n > 1 and < t < c, E[exp(i£ n )| fn-i] = exp(X„_iL(t)) 
which implies that 



(E.2) 



E[exp(tM n -L(t)5 n _i)] = 1. 



We are in position to prove (5.13). For all x > 0, let Z>„ = {\m n — m\ > x}. 
We have the decomposition D n = U D~ where = {fh n — m>x} and 
D~ = {fh n — m < —x]. By Holder's inequality together with (E.2), we have 
for all < t < c and q > 1, 



F(D+)<E 
(E.3) <E 



exp( -M n 



tx 



5*n- 



cxp 



(- 



Mr, 



L{t) 



S n -i^j expQ(L(t) - tx)S n -^j± D + 



< E 



(P 
-(L(t) - tx)S n - 1 
q 



i/p 



Taking the infimum over the interval [0, c] , we infer from (E.3) that 
(E.4) P(Z>+) < (nexp(-(p-l)I(x)S n ^)]) 1/p . 

Along the same lines, we also find that 
(E.5) F(D-) < (E[exp(-(p- l)I(-x)S n -i)])^ p . 

Finally, (5.13) immediately follows from (E.4) and (E.5). 

APPENDIX F 

This appendix is devoted to some justifications about the examples of 
random variables heavy on left or right. Consider an integrable random 
variable X with zero mean and denote by F its cumulative distribution 
function. Let H be the function defined, for all a > 0, by 



H(a)= P F{-x)-{l-F{x-))dx. 
Jo 



We already saw that X is heavy on left if, for all a > 0, H{a) > while X 
is heavy on right if, for all a > 0, H[a) < 0. Let Y be a positive integrable 
random variable with mean m and denote 



X = Y-m. 
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Discrete random variables. Assume that Y is a discrete random variable 
taking its values in N. For all n > 0, let 

n 

k=0 

After some straightforward calculations, we obtain that, for all a > 0, 

[m+a] 

H(a) = -a+ Sk-S[ m+a ] + {m + a}s[ m+a] -{m-a}s[ m _ a] 

k=[m—a] 

where, for all i£K, [x] stands for the integer part of x and its fractional 
part {x} is given by {x} = x — [x] and, of course, s n = for all n < 0. 

Continuous random variables. Assume that Y is a real random variable 
absolutely continuous with respect to the Lebesgue measure. Denote by g 
its probability density function. It is not hard to see that, for all a > 0, 

ram rm+a 

H(a) = —a + 2a / g(x)dx+ / (m + a — x)g{x) dx 

JO J a m 

where a m = inf{m — a,0}. Consequently, in order to check that X is heavy 
on left, it is only necessary to show that, for all a > 0, H (a) > 0. 
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